Human-Level Psychometrics for Cognitive Architectures

As artificial intelligence approaches the human level, the task of designing cognitive tests and challenges for robots becomes critical for clarifying and reproducing elements of the “magic” of human cognition. In solving this task, the store of knowledge of experimental psychology cannot be ignored. At the same time, methods and test paradigms used for evaluation of humans and/or brain models need to be adjusted before they can be used for evaluation of computational agents. The present work introduces new test paradigms and metrics that are adequate to assess higher cognitive features in expected computational agents, while at the same time allowing for comparison with humans. Index Terms – Cognitive evaluation, psychometrics, intelligent agents, cognitive systems, human-level intelligence, machine consciousness, emotional intelligence, IQ, g factor, Turing test.